Lancaster
We Still Don't Understand High-Dimensional Bayesian Optimization
Doumont, Colin, Fan, Donney, Maus, Natalie, Gardner, Jacob R., Moss, Henry, Pleiss, Geoff
High-dimensional spaces have challenged Bayesian optimization (BO). Existing methods aim to overcome this so-called curse of dimensionality by carefully encoding structural assumptions, from locality to sparsity to smoothness, into the optimization procedure. Surprisingly, we demonstrate that these approaches are outperformed by arguably the simplest method imaginable: Bayesian linear regression. After applying a geometric transformation to avoid boundary-seeking behavior, Gaussian processes with linear kernels match state-of-the-art performance on tasks with 60- to 6,000-dimensional search spaces. Linear models offer numerous advantages over their non-parametric counterparts: they afford closed-form sampling and their computation scales linearly with data, a fact we exploit on molecular optimization tasks with > 20,000 observations. Coupled with empirical analyses, our results suggest the need to depart from past intuitions about BO methods in high-dimensional spaces.
Bayesian Inference for Left-Truncated Log-Logistic Distributions for Time-to-event Data Analysis
Mostafa, Fahad, Haque, Md Rejuan, Rahman, Md Mostafijur, Nasrin, Farzana
Parameter estimation is a foundational step in statistical modeling, enabling us to extract knowledge from data and apply it effectively. Bayesian estimation of parameters incorporates prior beliefs with observed data to infer distribution parameters probabilistically and robustly. Moreover, it provides full posterior distributions, allowing uncertainty quantification and regularization, especially useful in small or truncated samples. Utilizing the left-truncated log-logistic (LTLL) distribution is particularly well-suited for modeling time-to-event data where observations are subject to a known lower bound such as precipitation data and cancer survival times. In this paper, we propose a Bayesian approach for estimating the parameters of the LTLL distribution with a fixed truncation point \( x_L > 0 \). Given a random variable \( X \sim LL(ฮฑ, ฮฒ; x_L) \), where \( ฮฑ> 0 \) is the scale parameter and \( ฮฒ> 0 \) is the shape parameter, the likelihood function is derived based on a truncated sample \( X_1, X_2, \dots, X_N \) with \( X_i > x_L \). We assume independent prior distributions for the parameters, and the posterior inference is conducted via Markov Chain Monte Carlo sampling, specifically using the Metropolis-Hastings algorithm to obtain posterior estimates \( \hatฮฑ \) and \( \hatฮฒ \). Through simulation studies and real-world applications, we demonstrate that Bayesian estimation provides more stable and reliable parameter estimates, particularly when the likelihood surface is irregular due to left truncation. The results highlight the advantages of Bayesian inference outperform the estimation of parameter uncertainty in truncated distributions for time to event data analysis.
Meet the young team of software engineers slashing government waste at DOGE: report
Fox News host Laura Ingraham gives her take on the spending freeze on USAID on'The Ingraham Angle.' Tesla and Space X CEO Elon Musk's DOGE efforts to slash government waste and streamline the federal bureaucracy include the hiring of several up-and-coming young software engineers tasked with "modernizing federal technology and software to maximize governmental efficiency and productivity." Six young men between the ages of 19 and 24 -- Akash Bobba, Edward Coristine, Luke Farritor, Gautier Cole Killian, Gavin Kliger and Ethan Shaotran -- have taken up various roles furthering the DOGE agenda, according to a report from Wired. Bobba was part of the highly regarded Management, Entrepreneurship, and Technology program at UC Berkeley and has held internships at the Bridgewater Associates hedge fund, Meta and Palantir. "Let me tell you something about Akash," Grata AI CEO Charis Zhang posted on X about Bobba in recent days. "During a project at Berkeley, I accidentally deleted our entire codebase 2 days before the deadline. Akash just stared at the screen, shrugged, and rewrote everything from scratch in one night -- better than before. We submitted early and got first in the class. I trust him with everything I own."
Fox News AI Newsletter: Musk vs. Altman
Elon Musk, right, has cast doubt on whether there is enough funding available to follow through on a 500 billion AI infrastructure project announced by President Donald Trump on Tuesday. OpenAI CEO Sam Altman, left, pushed back on Musk's claims. EMPTY COFFERS?: Business magnate and X CEO Elon Musk has cast doubt on whether there is enough funding available to follow through on a massive 500 billion artificial intelligence (AI) infrastructure project announced by President Donald Trump on Tuesday. SpaceX and Tesla founder Elon Musk speaks during an America PAC town hall on Oct. 26, 2024 in Lancaster, Pa. ( Samuel Corum/Getty Images) ON THE BRINK: Walter Isaacson, author of "Elon Musk," discusses the Trump administration's collaboration with tech giants to drive AI innovation and technological advancement on "America's Newsroom." CONTROVERSIAL TECH: Artificial intelligence (AI) tools are now available for future medical professionals at one Texas university to navigate the complexities of pregnancy and abortion--a development that further blurs the line between technology, politics and healthcare.
Fox News AI Newsletter: OpenAI responds to Elon Musk's lawsuit
Raj Goyle, CEO of intelligence firm Bodhala and former Democratic Kansas state representative, told Fox News Digital it is encouraging to see members of both parties come together to try and determine the source of these drones. SpaceX and Tesla founder Elon Musk speaks during an America PAC town hall on October 26, 2024, in Lancaster, Pennsylvania. AI WARS: OpenAI is pushing back against Elon Musk's latest attempt to rework his lawsuit against the artificial intelligence giant that seeks to prevent the company from moving to a for-profit structure, noting in a blog post and legal filing that Musk had argued for it to do so years ago. AGE OF AI: OpenAI CEO Sam Altman is joining the list of U.S. tech titans donating to President-elect Trump's inaugural fund, a spokesperson exclusively told Fox News Digital. ARTIFICIAL INTELLIGENCE: The House task force on artificial intelligence is urging the U.S. government to aim for "a flexible sectoral regulatory framework" for the technology in a nearly 300-page report released Tuesday morning.
AMUSD: Asynchronous Multi-Device Speculative Decoding for LLM Acceleration
Large language models typically generate tokens autoregressively, using each token as input for the next. Recent work on Speculative Decoding has sought to accelerate this process by employing a smaller, faster draft model to more quickly generate candidate tokens. These candidates are then verified in parallel by the larger (original) verify model, resulting in overall speedup compared to using the larger model by itself in an autoregressive fashion. In this work, we introduce AMUSD (Asynchronous Multi-device Speculative Decoding), a system that further accelerates generation by decoupling the draft and verify phases into a continuous, asynchronous approach. Unlike conventional speculative decoding, where only one model (draft or verify) performs token generation at a time, AMUSD enables both models to perform predictions independently on separate devices (e.g., GPUs). We evaluate our approach over multiple datasets and show that AMUSD achieves an average 29% improvement over speculative decoding and up to 1.96$\times$ speedup over conventional autoregressive decoding, while achieving identical output quality. Our system is open-source and available at https://github.com/BradMcDanel/AMUSD/.
2 Partial Models from a finite set A and the environment (stochastically) emits an observation o
This paper introduces timeline trees, which are partial models of partially observable environments. Timeline trees are given some specific predictions to make and learn a decision tree over history. The main idea of timeline trees is to use temporally abstract features to identify and split on features of key events, spread arbitrarily far apart in the past (whereas previous decision-tree-based methods have been limited to a finite suffix of history). Experiments demonstrate that timeline trees can learn to make high quality predictions in complex, partially observable environments with high-dimensional observations (e.g. an arcade game).
StitchNet: Composing Neural Networks from Pre-Trained Fragments
Teerapittayanon, Surat, Comiter, Marcus, McDanel, Brad, Kung, H. T.
We propose StitchNet, a novel neural network creation paradigm that stitches together fragments (one or more consecutive network layers) from multiple pre-trained neural networks. StitchNet allows the creation of high-performing neural networks without the large compute and data requirements needed under traditional model creation processes via backpropagation training. We leverage Centered Kernel Alignment (CKA) as a compatibility measure to efficiently guide the selection of these fragments in composing a network for a given task tailored to specific accuracy needs and computing resource constraints. We then show that these fragments can be stitched together to create neural networks with accuracy comparable to that of traditionally trained networks at a fraction of computing resource and data requirements. Finally, we explore a novel on-the-fly personalized model creation and inference application enabled by this new paradigm. The code is available at https://github.com/steerapi/stitchnet.
Human Autonomy as a Design Principle for Socially Assistive Robots
High levels of robot autonomy are a common goal, but there is a significant risk that the greater the autonomy of the robot the lesser the autonomy of the human working with the robot. For vulnerable populations like older adults who already have a diminished level of autonomy, this is an even greater concern. We propose that human autonomy needs to be at the center of the design for socially assistive robots. Towards this goal, we define autonomy and then provide architectural requirements for social robots to support the user's autonomy. As an example of a design effort, we describe some of the features of our Assist architecture.